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ABSTRACT , , ^ - 

^cr-^ s Identifyiig the effects of-the New York State 

Eiperiaental -Prekindergart,en Program on the development of chil^dren * 
requires that a broad arr,ay of data be analyzed, fn study4.)ng the 
effects of the program, the evaluation unit is interested in 
determining^ whether , there is an overall ^f feet, and whether children 
of different ages and different levels of^ initial devplopmeiit are 
affected. in the same way. An illustration is. given to. clarify bow the 
proctes:? of hypotheses testing reduces the num|)er of assumptions one 
has^o accept in drawing conclusions about the progranh-ef fects. The • 
analysis of .covariance procedure illustrated involves- 18 possible 
hyeotlieses; however, the Kr,anching nature of t'he analysis results in 
^the testing of no more than siif hypotheses. The resulV obtained' in 
testing a gi.ven hypothesis leads to another hypothesis while- 
siaulat aneously precluding other .hypotheses. Thus, effici^ency of. 
de&i5[n is achieved by, consideic^iM a large number of hypotheses^ while 
testing only ? limited i^umb^ir of ^hem. At the same t}.me, the design 
'makes it possible to retain' the most general hypothesis which 
adequately explains the^data. (Author/H?) / v ' . 
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HYPCniESIS, TESTING IN EVALUATING A' 



; - ^ , PREKINDERGARTkN PRbcM>J^' 1LL\)STRATIVE' PROCEniJRRS 

■ . . ■ ■ • ' . ^ 

Identifying the effects of ISxperimental Pr^kindergarten Program on the * 

development of children tequires that a broad array of data be analyzed, 

^ The simplest analysis would l^e to compare scores of a group of chi\dr<}n on 

' a mcasare of development at the beginning of their prekindergarten year wl>di 

scores 'on the same measure at the end of the year. Obviously, this procedure 

leaves much to bo desired since it does not provide any way of accounting for 

Other posBibfc influences on the children U .^eyeloptnent. To' acce'pt the results 

< . * V ^'^ ' ^ ^ - . 

v of this^ simpler design as an indication of ¥he effects pf the prekindergarten 

progranT requires that, on<2 also^aceept a number of assumptions, including 

' \ ' • / ^ 

assumptions that sucrh factors as maturation, family conditions, and general 

aMlitA^ bf.the children do not affect the performance of the children on the ^ 
\ • -/ ' ^ . ' " - . ' 

' . ' ""''-^ . - ^ • 

measure being us"ed. In fact, this simple design requires.^one to accept such a 

vast number of assumptions that they cbald be summed under on?' general assumptidn: 

\ - * ' * * ^ ' ' ' * i^'^^^^j*^^ 

. Nothlnc in the world'outside the prekindergarten program affects the,,^^elopment , . 

of children" on the measure in question. For purposes of makling decisions about 



a prog^ramji thi^ degree of uncertainty about the, possibl^^lftf luence's on children's 
" development is .unsatisfactory. It may also^Jj^' unj»6cdssary. 

While it is not possible to elimi;ciate a^^df assumptions, a large number of 
' ^them can 't>c brought'out of , the^re^orfm of^^eculation. Tlie data whiqh-can be 



obtained and the naturc.^4^ tVffe\anal-ytical design will determine ^ich assumptioas 



can- b^ tesf'fed andj^c>^^u8t remi^Lnla TJie aim-6f research is^ to reduce 
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both' the number and thc'logicnl Importonce of the a'ssumptions that remain unoxalWTrre 
so that <:he consumer of the research findings has to |ccept as Bttle as possible ... 

%Sevcral stops* are followed in Examining a. group of assuinptipifs : , They are ♦ 
Btatcd'osnuH hypotha'sos ; 'data are collected on variables referred to in the 
- •hypotheses; and statistical tcjsts are carried out to determine whether thej^ta 
warrant retaining 'or T;^5jecting each .hyp6.|iie sis , 

An illustration. may clarify how the process of testipgrt^ypotheses reduces 
. the number of assumptions one has to accept in dr^jwllfg conclusions about the 
effects of a program. • 

In studying the effpcts of the Experimt2n'td 1 PrekindeTgarten^Programi' the 
■ V evaluation unit \s Interested not only in^ddtermining whether thcpre Is an over- 
nil of feet but nlHO whether it affects children^of different ages and .different ■ _ 
■ levels of ;irJTtt>al d.avelopm(int in the same" way. /In other wo'rds, it is not. ■ ^ 

-nctesstfry to assurfie^ that the program affects children of different ages and 
lovols of~.devolopinent in the same way, since these assumptibtis are testable and . 
could hflve meaning for the wpy 'in which the program is carried out. Nor is it / 
n^ssary to assume that age and level of development do not interact in^some^^ ^ 
fashi*on, since that assumption^ is also testable and could have implications for \ 
the program. * ' . * • . 

fasting these hypotheses is an integral part pf examiriing Question 1 tn the" 
Evaluation Design* for the Experimenta^S^ekindirgarten Pfograiji. Question 1 asks: 

' ^ * ^: ' I " ' . 

Do Prckindorgarten ^groms have a measurable effect <ipbn child^ren's cognitive 

■ ■ ^ ' ; ■ " . 

and nftncognitive development between the time 'they enter and leave t^he progranj? , 

> . • \ '■ . . ■ / , • ' ' 

' . *F:vn iri.-itlon Dcaign: Now York State Experim^n ta 1 Prekind^rgarten Progran),. • 
Albany New York: The State Education Department", Bureau of-€hild Development 
• and Parent Education, Bureau of School Program^ Evaluation. 'September, 3, J975. . 
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* The following is oneof ^many possible analytic procedures for answering the 
stlon. . It is presented ^rc^ to illustrate the kind of analy^'s whic^ can be 

used to study'this and' other c/uestions about the Prekindergarten Prograra. In sf 

order to ^eep this example rUZatively siimpl^e, it is assume^ that*any eifex^s of 

-control variables are linearA 
As will be seen, this 

the branching nature o^ the 



anal^is involves 18 possible^ypoth^es ; hc^ever 
anal^^is* results in the, test'ing of no more t^n six 
hypotheses. /Thej^esult dbt^ained/'in teTElftg,.3-^iven hypothesis leads! to an\t:'h€!r 
hypothesis while s,imultaneously precluding other\hypotheses • Thus, . etff icienfcy 
of design is achie7ed by considerin^a large number of hypotheses whil^ being 
required to test only a liiuited number of them. At the same time , the fdesign 
^makes it possible to^retain the most general hypothesis \A\ic\i adequa^'eA explains 

the data. ^ ; ' . 

The example described beloiw follows arnalysis_9f covariance procedurj^s. The 
variable's used^are : * Walker posttest Ccritcr^ion) ; Walker pretest and chiiidren's 
ages at entry to the program (control varia'bles) ; and Prokindergarten experience 

^ ^ . . ' ' ^ . - ^ ^ /' ^ ' 

( treatm'ent ) , • ' ^ ^ * 

The hypotheses a^e presented in Table 1. 

At thexjutset, the example' ^eeks to establish the usefulness of the control 
variables (Hypotheses 1 and 2)^^^^ontrol V£iriai)les are dropped if they .do not 
contribute to an understandwig^f a Prekindergartcn effect. This is the case^ 
at Hypothesis, 3 which i^ answc^c)d by the familiar t-lest,' If tlie t-test is ^ , . 
significant, the Jjv^ot|xcsi s is-rejecLed and one may coivclude that there is a 



Prqkindergarp^ effect*^ 




-A 



.1 Table 1 



/ 

^ Illus'trative Analysis of Covariance 
- with two Control Variables- C^llcer and Age) 



Hyp. .1: ..No ^.ffect on the Ua^ll^er pretest within age for either, the ekperimental . . 
' ^ ' or the control group/ . ~ ^ \ ' -q > 1' 

' ^Retain go to 2 - ^ ' • 

Reject go to 6 , 

Hyp. .2: ' No effect of age^in ei^r th^ experimental group. or the control group.^ 

Retain go to\3 

Reject go*^to ^N- ' ' ' - ' 

Hyp. > 3: No ep^ct of Prekindergarten Program (.experimental mean compared to , ^ 
, ' * conirrol mean). , f i ^ 

^ / Retain - Sttjp 

' Reject - Stop - 

Hyp. A: No'Ptekindcrgarten by age irtteraction. ^ ^ ^ 

Retain ga to " ^ < 



Reject - Stop 



^'Hyp. 5: Preki/idergarten effect wit^ control' for age. 
'I ^ ' Retain - Stop ^ 
. ^ ' . '"Reject - Stop - 

Hyp. e^TN^o Walker pretest by age' irjterafction; 

Retain go to 7 ' - j ^ 

^ ^ Reject go to lA ' , y 

Hyp. 7> No" Prekindergarten effect by Walker, pretest interaction, 
""'v. ^ J ' ' Retain go tQ 8 « . ^ ^ 

Re^xsct go to' 12 , ' ' . 

. Hyp. 8: No effect. of age with control'^or Walker pretext. 
• ' . Retain go t;o, 9 ^ ' V ^ 

^ 4: • *• ^ ^ Reject go to 10 • ; '* 

Hyp. 9:- No Prekindergarten effect with control, for Walker- pretest . 
I • Retain - Stop^ " . 

* ' Reject - Stop ^* 



Tabl*^ 1 (Continued) ' * 

*Hyp. 10:' l'roklnc)orgflrtro)'td:ftjct byvoge interaction In Sho presence of a 
homogeneous ^nlkcr pretest offoct. - . ^, . ' ' 

Retain go to 1.1 ^ ^ 

, . - Reject - Stop , * • 

Hyp. M:, No- 5rokindergarten effect at alT levels of ago and/Valker pretest. 

Ret'ain - Stop 
- ' ' Reicct -"^top 

Hyp* 12: No age effect in the presence of a treatment pretest interaction,. 

Retain - Strop \ 
' Reject go to 13 , . ' ' ^ 

Hyp. lily No Prekindergarten effect by age interaction ih the presence- of 
, treatment -by Walker pretest interaction. 

^ Rj^tain - Stop C ' \ 

Reject - Stop ^ . , ' • 

Hyp. 14: No interaction of the Prekindergarten effect ahd age and WalkeiT pretest. 

Retain go to 15 ^ , I • ^ 

Re j^cct' *• Stop * ^ ' . \ * . ' 

Ifyp. 15: No Prc^klndnrgnrton rffoct by Walker pretest interaction. ^ 

Rotain go to 

"Reject go to IB * • ^ 

- Hyp. 1-6; No Prekindergarten of£oct by age> interaction in the presence of Walker 
pretest by- age interaction. 

* kc'taih go to 17* 
Reject - Stop J • 
\ , * * 

Hyp.' 17: No Prekindergarten effect at all levels JFfrage and Walker, pretest in* • 
• the presence of WaTker pretegt by a^ge interaction.* 

. Retain - Stap, ' ' ' ^ ■ 

„ . Reject - Stop 



Hyp. 18: No Pre-kindergarten effect by age interaction ip the presence of pretest 

^ interactiofis. _ . ' . ^ 

. ^ ' Retain Stop ' 

• ' Reject -.;Stop ' ^ . 



ERIC . . . - ^ o 



- The case where one control variable (age) hoe a simple effect andjihe other 
(Walker pretest') h.as hone at all, is given^at Hypothesis 5. Analysis for this 
hypothesis is again a familiar an/ilyais of -covariance with one control variable. 

Since this -a>»aly^isl^iypc^thcsi8 s5) was p^uccd only after tostinjg for a ''alker 

\ \ - * ' " * * 

pretest effect, the rWults^^of this analysis, and the possible conclusion that 

prokindorgarten has^.^nVff^c^ hold oyer the entire ran^e of the. rejected^ 

variable (Walker Pretest])^.' Th^ corresponding hypothesis, where' Ikqr pretest 

is retained as a control variable while the age variable was eliminated, is 

given in Hypothesis 9. \ • -s 

< \ 

Complex effects af Prekindergarten are located in interaction hypotheses. 
The most complex, of course, is the triple intei:ax:t:ion, expressed in Hypothesis 
14., A STOP signal is given after some of these complex effects even though 
additional hypotheses could be tested. . ,^0'* ' 

In g(>neral, a STOP signal also design'Stes a point at which concluflons may 
be drawn. SQme of these Have been mentiqn^^ above in relation to Hypotheses 3 
and 5.' A significant^ finding at Hypothesis 14, which'^ would cause rejecti^on of 
the hypothesis, would lead to rconclusions that the Prtkindergarten effect was' 
dlffercht for children of different ages and of .different score levels on the 
protest. -It might be possible to plot this to show that Prekindergarten was 

most effective for children enterlftig the program between certain ages and 

— . - * 

'scoring bc^twjeen certain raw, score points on the Walker pjetest. The data^ 'of 
* course, would determine the precise naturi^f such a conclusion, / 

Thus, the results of the analyses may help answer questions of differing 
breadth of impact fr^om ^^Should the program be continue'd7*Vthrougii "How can the 
guidelines of the program be changed to make the program more effective?" to 
"What specific practices are njost effective with which childrqn?^' 



* *Why is a complex ano,lysis-*rcqulrcci' to anuwer a rather straight ^ forward 
■question such as Quostion 1? • ^ ' ' 

Attempting to answer the general question of .effects of the Prokindcrgorten 
Program without considering many other possible iafluCacqs on chiltlrcn's develop- - 
mcnt. could lead to on incorrect answer. Differences^ resulting from other in- ' 
flucncos could be attributed to the Prckindergartcn Program, with the' result^ 
that lt;,.ls incorrectly concluded that thc^ |>a:ogrom .has been effective. Or other 
influences could-obscure the rral effects of the Prekindergartcn Program so that 
it .is incorrectly concluded that the program has not been^ef f ectljve; this is^ 
likely to happen if, for 'example, the program ^is more' effective for chi Idxren^ of 
a given age or^^a given level ofUnitial performanc^e. than it is for children of 
other ages or performance levels ^ ■ ^ 

But assuming r-that. the general quast,ion is answered fiorrectly, it is important 
to know how specific factors affect different children under different circum- 
stances. It is possible for a. program to be generally ineffective but for 
specific aspects of the program t6 be effective; or for the program to bo 
effective with some children but not^ with others. Tiven with a i/rogram which has 
been found to be effective overall, it is unlikely that altaspects of ,the program 
are equall/y effective with all children. Efficiency can be improved by an. analysis 
►f proftrJ^m variations in relation to chi Idren ' s^hciracteristics. ^ 



r 

I A great variety of data is being coUc/tedoii the Experimental Prckindergartcn 

Prtegram to make A possible to carry out analyses similar to that described above, 

\ ^ . ' -\ 

Throughout the analyses, an effort will be made to obtain meaningful answers to 

the research questions *wh.ile^doJ.ng justice to the complexities of the data and the 
phenomena they represent, 



